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ABSTRACT 



The study reported in this paper is designed to provide 
evidence of the convergent and discriminant validity of the Flexible 
Combination Test (R. Dillon) . Two premises were tested: (1) flexible 

combination accounts for significant variation in cumulative grade-point 
average, high school rank, and cumulative Act Assessment scores; and (2) the 
importance of flexible combination ability to academic success varies as a 
function of the type of academic abilities being measured. Data from 99 
undergraduates indicate that flexible combination ability predicts each of 
the criterion measures. In addition, data indicate that the relationships 
between cognitive flexibility and academic success vary as a function of 
academic subject areas. Flexible combination ability is more important to 
achievement in mathematics and English than natural science or social 
science. Data from this study provide further evidence of the importance of 
flexible combination ability to academic success and point to the importance 
of including cognitive flexibility in models of intelligence. (SLD) 
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Abstract 



The study reported is this paper is designed to provide evidence of convergent and 
discriminant validity of the Flexible Combination Test. We test the premises that (a) 
flexible combination accounts for significant variation in cumulative grade-point average, 
high school rank, and cumulative ACT, and (b) the importance of flexible combination 
ability to academic success varies as a function of the type of academic abilities being 
measured. Data indicate that flexible combination ability predicts each of the criterion 
measures. In addition, data indicate that the relationships between cognitive flexibility 
and academic success vary as a function of academic subject area. Specifically, flexible 
combination ability is more important to achievement in mathematics and English than 
natural science or social science. Data from this study provide further evidence of the 
importance of flexible combination ability to academic success and point to the 
importance of including cognitive flexibility in models of intelligence. 
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Convergent and Discriminant Validation of Flexible Combination Ability 

The importance of flexible thinking and problem solving are clear, as educational 
and professional demands change at an accelerated rate in our society. Professional and 
practical life events often require that individuals be able to combine stimulus or situation 
elements in more than one way to arrive at a correct solution to a problem. 

Dillon’s multiple component information-processing model is an elaboration of 
Sternberg’s (1985) triarchic theory. Dillon’s model is comprised of six components: (a) 
encoding; (b) rule inference; (c) rule application; (d) confirmation; (e) learning; and (f) 
flexibility. Flexibility, in turn, is comprised of three components: flexible encoding; 
flexible combination; and flexible comparison. 

"A major hallmark of cognitive development is the ability to apply strategies 
flexibly and appropriately." (Plumert, 1994) The theory of cognitive flexibility is not a 
new one: 

It has long been recognized that the ability to vary one's mode of approach to 
solving complex problems is an important factor in achieving a successful 
outcome. Being able to view a problem from a flexible stance, in which 
alternative options are freely considered, is an important feature of intelligent 
behavior, and is particularly important in higher order cognitive functioning. 
Cognitive flexibility is thought to be important because it allows the human to 
consider a variety of behavioral options or solutions prior to executing an overt 
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response. In this fashion, overt trial and error behavior is reduced to some 
minimum level. (Cosden, 1979) 

Though most cognitive psychologists agree that some form of flexibility exists, 
often their respective definitions differ. Murray (1990) reported that flexibility was the 
ability to not only perceive interrelationships between concepts but also to understand the 
important distinctions between them. In other words, flexibility is the ability to see the 
similarities and differences between things. This definition is slightly different than the 
one offered by Spiro in 1991. Spiro believed that 

cognitive flexibility includes the ability to represent knowledge from different 
conceptual and case perspectives, and then the knowledge must later be used to 
construct from those different conceptual and case representations a knowledge 
ensemble tailored to the needs of the understanding or problem solving situation 
at hand. (Spiro, 1991) 

The major distinction between these two definitions is that Spiro believed that the 
insight gained from flexibility later must be utilized, and that it is specific to the situation 
at hand (Spiro, 1991). It is this type of applied flexibility that Dillon had in mind when 
she generated her three component model of flexibility. Dillon defines flexibility as the 
ability to define stimulus attributes in more than one way, the ability to generate 
more than one tactic in solving tasks or items that can be solved in more than one 
way, and the ability to change tacticswhen the demands of the task change. 
(Dillon, 1992) 

Dillon’s (1992) cognitive flexibility model decomposes aspects of cognitive 
flexibility that are confounded in other flexibility work. The flexibility model is made up 
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of three subcomponents: (a) flexible encoding, (b) flexible combination, and (c) flexible 
comparison. This three component division of flexibility is a departure from the other 
researchers in this area. Most researchers confound the three components of flexibility, 
using only one predictor in their models. However, their definitions encompass all of 
Dillon’s separate components. (Cosden, 1979, Murray, 1990, Spiro, 1991) Dillon 
contends that each component should be viewed individually. 

Dillon’s first component, flexible encoding , taps an individual’s ability to encode 
multiple meanings for each stimulus. The task stimuli require examinees to construct 
multiple spatial forms using multiple definitions of each stimulus. The second 
component, flexible combination, assesses an individual’s ability to generate multiple 
tactics for solution for a range of inductive reasoning items. The third component, 
flexible comparison, is developed to assess an individual’s ability to change solution 
tactic as the demands of the complex reasoning task change. Here, examinees solve a set 
of items of the same type and then switch to a different type of item. Each component of 
Dillon’s cognitive flexibility model has been found to be a significant contributors of 
academic success. (Dillon, 1 992) 

The work reported in this paper has three objectives. The first objective is to 
provide evidence of the psychometric credibility of Dillon’s Flexible Combination test. 
The second objective is to provide evidence of criterion-related validity by testing the 
premise that flexible combination ability accounts for significant variation in concurrent 
and postdictive indices of academic achievement. The third objective is to provide 
evidence of convergent and discriminant validity by examining patterns of relationships 
between flexible combination and different subtests of the ACT. Specifically, we test the 
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premise that the importance of flexible combination ability to academic success is 
greatest in mathematics, then English, then natural science, then social science. 

Method 

Participants 

The sample was comprised of 99 upper-division, college undergraduates. Males 
and females were approximately equally represented. 

Instruments 

Twelve complex figural analogies, taken from the Advanced Progressive Matrices 
(APM; Raven, 1965), were used as test stimuli. Each item could be solved in at least 
three ways to yield the same correct answer. Figure 1 depicts a sample item. 



Insert Figure 1 about here 



In addition, cumulative grade-point average (GPA), high school rank, and ACT 
Composite data were tabulated. 

Procedure 

Examinees were instructed to solve each item, generating a written protocol 
during item solution. Subsequently, examinees were instructed to solve the same item — 
using a different tactic to arrive at the same correct answer. Examinees were instructed to 
generate as many solution protocols as possible for each item before proceeding to the 
next item. Examinees were required to attempt all 12 items. 
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Results 

Data were analyzed in three phases. A .05 significance level is used in all 
analyses. First, internal consistency reliability was computed for the total test using 
Cronbach Alpha. Data indicate a Cronbach Alpha of .76 for the present test 
administration. Using the Spearman Brown Formula (Allen & Yen, 1979), data indicate 
that, if the test were increased to the full length of the APM, reliability for this test 
administration would be .927. 

During the second phase of analysis, data indicate that flexible combination 
ability predicts cumulative GPA, F(l, 98) = 5.55, p < .05. In addition, flexible 
combination ability accounts for significant variance in high school rank, F(l, 52) = 
22.47, p < .001 . Similarly, flexible combination accounts for significant variance in ACT 
Composite, F(l, 67) = 31.88, p < .001. 

During the third phase of analysis, data are analyzed to test the premise that 
flexible combination ability is more important for success math and English subtests of 
the ACT than for natural science and social science subtests. When ACT math is 
excluded from the model, the reduction in R 2 is significant, F(l, 63) = 9. 14, p < .001 . 
When ACT English is excluded from the model, the reduction in R 2 is significant, F(l, 
63) = 2.49), p < .05. when ACT Natural Science is excluded from the model, the 
reduction in R 2 is not significant, F(l, 63) = 1.07, p > .05. When ACT Social Science is 
excluded from the model, the reduction in R 2 is not significant, F(l, 63) = .13, p > .05. 
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Discussion 

Flexible combination ability is an important predictor of concurrent and 
postdictive measures of academic achievement for the present sample of college students. 
The consistent pattern of results across the three criterion measures reflects the robustness 
of cognitive flexibility. Evidence of convergent and discriminant validity of this test 
administration comes from the pattern of data when flexible combination ability is 
validated against ACT subtests. The relationship between flexibility and mathematics 
achievement is greatest because the reasoning strategies used to solve the figural 
analogies are logico-mathematical in nature. Flexible combination also is an important 
predictor of English achievement because most people represent figural analogies 
verbally in addition to figurally. 

Taken together, the data indicate that flexible combination ability is an important 
source of individual differences variation. Models of intelligent performance well might 
include measures of cognitive flexibility. 
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Figure 1. 

Sample Item From the Advanced Progressive Matrices 
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